Stop autosave from stalling the game at high speed - #260

At 8x, 13x and 40x the game runs smoothly for a second or two, then freezes for a fraction of a second, over and over. The freeze is the autosave. Every 256 ticks GameGUI::syncStep serialized the whole game, including every cached pathfinding gradient, and wrote it to Auto_save.game on the game thread. Late-game saves reach 65–160 MB, and at 8x 256 ticks is only about 1.3 seconds.

Changes

1. Bulk gradient writes. Gradient fields go out as one run of bytes through a new OutputStream::writeUint16Sections. Each value used to cost three virtual calls, a name string the binary stream ignores, and its own 2-byte SHA1 update. The output bytes and SHA1 are identical; TextOutputStream keeps the per-value loop.
2. Autosaves are hashed on the writer thread. Game::save can leave the whole-file SHA1 for later, recording the hashed range and the header bytes as first written in a DeferredGameSHA1. The BackgroundFileWriter applies it on its worker just before the write, so an autosave's bytes, SHA1 included, are identical to an inline-hashed save, and the game thread does no hashing.
3. MapHeader::operator== compares the SHA1 the right way round. It has been inverted since 8cf81c9 (2008): std::equal(...)==0 was true only when the hashes differed. Engine::haveMap, its only caller, so re-downloaded identical files and trusted a different file with the same name, team count and map offset. A client joining a YOG game now keeps its local copy exactly when it matches the host's.
4. Autosave spacing follows the speed preset. Saves stay about 10 seconds of real time apart: every 256 ticks at 1x, 2,048 at 8x, and 10,240 at 40x or maximum.
   • Normal speed saves on the same ticks as before (79, 335, …).
   • Counting from the last save also stops a soft-paused game, stopped on a save tick, from autosaving every frame.
5. The disk write moves off the game thread.
   • Where the work happens: the game is still serialized between ticks, into memory. A new GAGCore::BackgroundFileWriter then replaces the file through FileManager::writeAtomically on a worker thread.
   • Overlapping saves: a snapshot that hasn't started writing is replaced by a newer one.
   • Thread lifetime: the worker exists only while there's something to write, so waiting leaves no thread behind and a forked process never inherits one.
   • Waits: the engine waits for a pending write before a session ends, before an in-game save, and when GameGUI is destroyed.
   • Faster memory stream: MemoryStreamBackend appends past its end instead of resizing, which zero-filled every byte before writing it. It also gains reserve() and takeContents(), so the snapshot moves to the writer without a copy.
6. Setting to turn autosave off: Settings → Gameplay → Autosave (autosaveGames, on by default), translated for every language.

For review: behavior changes

• Autosave cadence. At faster presets autosaves are fewer in ticks but the same in real time. A crash at 8x can lose up to ~2,048 ticks, about the same ~10 seconds of play as at 1x.
• The new setting is a player-facing feature.
• Joining a YOG game. With the comparison fixed, a client whose local file matches the host's skips the download, and one whose file differs downloads it. Before, both went the other way.
• Timing of the file update. Autosave now reaches the disk shortly after its tick instead of within it. The late-game writes took 30–70 ms median, up to 187 ms.

Compatibility

• Simulation: unchanged. gd-bigarena-long for 20,000 ticks ends on master's checksum, 5abcb33f. A 3,000-tick run ends on 2a2689ed with autosave on and off.
• Save format and fields: unchanged. The final background-written Auto_save.game is byte-identical to the one this PR's previous commit wrote synchronously.
• Replays: the 20,000-tick replay is byte-identical to master's. The autosave-on and autosave-off replays are byte-identical to each other.
• Network protocol: no message changes. The SHA1 comparison is local to the joining client, so an old client keeps its inverted comparison against a new host and vice versa; neither side's messages change.

Verification

All results are from this branch's final tree.

• Harnesses:
  • BufferedFileStreamHarness passes. It now also checks that MemoryStreamBackend overwrites, appends, zero-fills seek gaps and hands over its contents.
  • SavegameSafetyHarness passes, with new checks:
    • background writes keep the newest snapshot with its own finish step, finish on destruction and continue after a failed write
    • the background-written autosave, SHA1 included, matches an inline-hashed save written straight to a file
    • with a stale map offset, so the header backpatch rewrites hashed bytes, a deferred SHA1 still matches the inline one
    • Engine::haveMap trusts the saved file for its own header and rejects it when the host's SHA1 differs by one bit
    • with autosave turned off, the previous save stays untouched
  • Linux CI runs both harnesses; Windows CI runs the savegame harness.
• Settings: test/run-settings-tests.py and test/run-game-speed-tests.py --settings-only pass. SettingsScreenTest checks that the Autosave toggle persists off and back on.
• Translations: data/check_translations.py --strict gives the same report as before this change, and test/test_translations.py passes.
• Test suite: scons -C test builds, and TestsRunner (187 tests), WinningConditionsHarness and ReplayStepCounterTest pass.
• Server build: succeeds.
• Negative controls: restoring the inverted operator== fails the haveMap check; hashing the backpatched header instead of the recorded one fails the stale-offset check.

Test games

• games/gd-bigarena-long.game: Oazis, 256×256, 11 Castor/Warrush teams.
• A no-Warrush game on the same map:
  GLOB2_TEST_SEED=7 glob2 -test-games-nox 1 --map Oazis --matchup nicowar,econo,nicowar,econo,nicowar,econo,nicowar,econo,nicowar,econo,castor --save-game-as <file>

How the runs were done: release build, glob2 --nox <game> <ticks> 1, with a disposable HOME whose preferences.txt sets gameSpeed and, for the on/off runs, autosaveGames.

Game-thread stall per autosave

Method:

• Inputs: late-game autosaves of the two test games, each loaded and run for 35 seconds at 1x.
• Instrumentation: local tick profiling that isn't part of this PR.
• Excluded: the first autosave after loading, which has no size hint yet.
• Measured before hashing moved to the writer thread. The game thread did no hashing in either version, and now only additionally serializes the small map header twice, so these figures still describe it; they were not re-run.

• Previous-commit column: measured in an experiment build that ran both save paths on the same loaded saves.
• What the game thread still does: serialize the game into memory.
• Master, for scale: the stall averaged 260 ms (worst 435 ms) and 378 ms (worst 773 ms) over full 20,000-tick runs of the same two games. That is a different measure, so it isn't in the table.

Limits

• Remaining stall: each late-game autosave still holds the game thread for about 30–50 ms while it serializes into memory, roughly every 10 seconds of real time.
• Memory: a late-game snapshot, up to about 160 MB, stays in memory until it's written.
• At 40x and maximum: the 10,240-tick interval assumes the machine keeps up. A game that can't run that fast saves less often in real time.
• Translations: the two new setting strings are machine-drafted for all 33 languages. The Arabic help text was converted to presentation forms by script, to match that catalog. They need review by native speakers.
• Timing: measured on one macOS machine only.
• Interrupted writes: if the process dies while an autosave is still being written, that snapshot is lost. The previous Auto_save.game survives, because the file is only replaced by rename.
• Settings test in CI: CI doesn't run test/run-settings-tests.py, which covers the new toggle. It passed locally.
• Not tested: a manual two-client YOG join. haveMap is covered directly in the harness, not through a live join.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LPPkz7Vd7QLMsbQfqy3kHU